Instability of the topological order and localization of the edge states in HgTe 
quantum wells coupled to s-wave superconductor 
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Using the microscopic tight-binding equations we derive the effective Hamiltonian for the two-layer 
hybrid structure comprised of the two-dimensional HgTe quantum well-based topological insulator 
(TI) coupled to the s-wave isotropic superconductor (SC) and show that it contains terms describing 
mixing of the TI subband branches by the superconducting correlations induced by the proximity 
effect. We find that the proximity effect breaks down the rotational symmetry of the TI spectrum. 
We show that the edge states not only acquire the gap, as follows from the standard theory, but 
can also become localized by the Andreev-backscattering mechanism in a small coupling regime. In 
a strong coupling regime the edge states merge with the bulk states, and the TI transforms into an 
anisotropic narrow-gap semiconductor. 



I. INTRODUCTION 

A topological insulator (TI), a material in which the 
electronic spectrum possesses an energy gap in the bulk 
but has the special, so-called topologically protected, 
edge (surface) states falling into this gap, is onepoLthe fo- 
cal points of current condensed matter studies.EfQ Topo- 
logical insulators hold high technological promise since 
due to the ability of their topologically protected edge 
states to carry nearly dissipationless current they can be 
utilized in integrated circuits. Thus the question how 
robust the edge states are with respect to hybridization 
with the electronic states in the leads has become one of 
the focal points of current Tl-related research. 

Three dimensional (3D) TI have the Dirac spectrum 
with the finite mass in the .kutk- while the spectrum of 
the surface states is massless.oErtJ Two-dijpensional (2D) 
TI has the gapless helical edge states.tHlilO The surface 
states in 3D TI and the edge states in 2D TI are topolog- 
ically protected and are robust against all time-reversal- 
invariant local perturbations. □ It was shown experimen- 
tally that 2IX.IJ-state appears in HgTe/CdTe quantum 
wells (QW).By The TI in three-dimensional (3I)A-ma- 
terials was found in Bii_. r Sb; C , Bi2Se3 and so on.Lrt3 

Of special interest are the states that develop at the in- 
terface between a TI and an s-wave superconductor (SC), 
like, e.g., in Fig. [I], where the proximity effect gener- 
ates a superconducting pairing, it was shown that at 
the interface of 3D TI coupled to s-wave superconduc- 
tor p x + ipj,-superconducting state-appears but without 
time-reversal symmetry breaking .113 Numerical calcula- 
tions showed that the proximity of the superconductor 
leads to a significant renormalization of the original pa- 
rameters of the effective model (describing the surface 
states of a topological insulator.ll3 For 2D topological 
insulator coupled to s-wave superconductor it is known 
that the momentum-independent gap enters in the edge 



spectrum.E. 

Two-dimensional TIs have a unique property: their 
parameters can be tuned over wide ranges of their— val- 
ues by the appropriate choice of the QW width dJl3 In 
particular, the main parameter of 2D TI, the gap in the 
bulk spectrum, M, can be changed from zero up to room 
temperature energy scale. Phenomenological treatment 
of the proximity effect is based on the assumption that 
the gap M in the bulk spectrum of TI is much larger than 
the characteristics energy of the induced superconduct- 
ing correlations. Unfortunately numerical calculations 
do not give the answer how ths-spectrum of the TI-SC 
system develops in this regime. EJEil In what follows we 
will focus on the gaps M of the order of the energy of 
superconducting correlations. 

In this Paper we investigate the effect of superconduct- 
ing correlations on the topologically protected edge states 
and the bulk spectrum in the 2D topological insulator 
brought into a contact with the SC layer, see Fig. [I], 
and show the emergence of the SC correlations in the 
TI similarly to what was obser^ied-in GaAs containing 
a two-dimensional electron gas.E3'E3 Ordinarily, the ef- 
fective Hamiltonian of the TI in question is constructed 
from the symmetry considerations, see e.g., Ref. 17- 19 
and has the trivial structure of the induced supercon- 
ducting potentials. We demonstrate that the symmetry 
reasons suggest the additional non-diagonal (in the sub- 
band space) terms in the Hamiltonian. Moreover, using 
the microscopic tight-binding equations we derive analyt- 
ically the additional terms in the effective Hamiltonian 
of the TI describing coupling to the s-wave isotropic su- 
perconductor (SC) placed on top of it. These terms be- 
come especially important in the case where the bare gap 
parameter M of the TI becomes comparable to the char- 
acteristic energy of the induced superconducting correla- 
tions. We show, further, that the interplay of the super- 
conducting and "topological" interactions is essential and 
results in several effects, in particular, the collapse of the 
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topological order in TI. We find that while "topologicaly 
protected" edge states can ensure undisturbed propaga- 
tion of the charge (spin) carriers, superconducting cor- 
relations can block the edge current causing a peculiar 
localization effect. 



a) 



no edge states on this edge at low energies 




FIG. 1. (Color online) Sketch of the 2D topological insu- 
lator coupled to a superconductor. The thick arrows show 
schematically the edge states. 

On the qualitative level our results can be summarized 
as follows: in the absence of superconducting correlations 
the edge states spectrum is linear near the Fermi surface, 
comprising of two counter-propagating electron and two 
hole branches, respectively.B Importantly, the edge spec- 
trum is isotropic in a sense that it does not depend on 
the orientation of the edges with respect to crystallo- 
graphic axes, although the edge electronic states wave 
functions are orientation dependent. Namely, there is a 
phase difference between the wave function components 
corresponding to (E) or (H)-subbands in TI. The bulk 
spectrum is characterized by the gap M. At small SC- 
TI coupling (the quantitative criteria will be given below) 
the edge-states spectrum acquire a gap E s . Furthermore, 
superconducting correlations mix the subband branches 
and thus turn the resulting electronic spectrum in TI 
(edge and bulk), anisotropic, and E g starts to depend on 
orientation of the edge. This can cause localization of 
the low-lying edge states since an inevitable bending of 
the edge will create the regions along the edge were the 
edge-particle energy e < E g thus getting locked between 
the turning points where s = E g . At the turning points 
electron and hole excitations undergo Andreev reflection 
and form the localized Andreev bound edge states, see 
Fig. ^|a. As the characteristic energy of the induced pair- 
ing amplitude exceeds M then the gap of the continuum 
bulk states of TI collapses and TI behaves like the highly 
anisotropic narrow-gap semiconductor. 



II. EFFECTIVE HAMILTONIAN 

The low energy Hamiltonian of the two-dimensional 
(2D) topak*ical insulator formed in the HgTe QW has 
the formfllif 




(1) 



e -- spin up 
h - spin down 
^\ e — spin down 



h - spin up 
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FIG. 2. (Color online) a) A sketch of localized Andreev 
edge states in TI. b) Excitation spectrum in the 2D TI with 
proximity induced superconducting correlations: the edge 
modes at one Tl-edge acquired the gap while the edge modes 
on the opposite edge remain gapless. Solid lines show the 
edge states and bulk spectrum boundaries in TI without su- 
perconducting correlations. The parameters are chosen as: 
mk F t 2 a /(2Trh 2 M) = 1, tb = y/\Bl\/\B + \t a and the orienta- 
tion angle tp = 0. 



where H = + di<r l , i = {1,2,3}; a 1 are the Pauli 
matrices acting in the subband (isospin) space; tk — 
C — Dk 2 . We choose the frame of reference so that 
d= (k x A,-k y A,M - Bk 2 ). Here A, B, C, D and M are 
material parameters. The lower block of the Hamilto- 

i(7y is the 



nian, H — p T H* p = eu — di(—k)<j' l : where p - 
metric tensor in the spinor space. The chosen represen- 
tation for % enables us to employ the machinery of the 
tensor bispinor algebra developed for Dirac Hamiltonian. 

To construct the convenient form of the Bogoliubov - 
de Gennes (BdG) Hamiltonian describing superconduc- 
tivity, we introduce the time reversal symmetry operator, 



T 



/0 -D 
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-fi ® i&2 C . 



(2) 



where C is the operator of the complex conjugation and 
fj, are the Pauli matrices acting in spin space. Then the 
time-reverse of the BHZ-Hamiltonian ([]]) is THT" 1 = 
H. The BdG Hamiltonian is 



H+U 
A+ 



A TI 

-n-tuf- 



(3) 



where A TI is the effective proximity induced supercon- 
ducting pairing matrix coupling the spin and subband 
spaces. The effective chemical potential shift appearing 
in the BCS theory has matrix form lA. There is no reason 



3 



k=-k\?>. 





FIG. 3. (Color online) The edge states at the TI boundary for 
a weak coupling where the edge states become gapped, a) Un- 
perturbed edge-spectrum, b) Edge states with the proximity- 
induced gap. The shift of the zero point reflects the difference 
in the original chemical potentials. 



to believe that the matrix structure of U and A TI is nec- 
essary trivial like, e.g., in Rcf. [l?] [l^; symmetry consid- 
erations in fact allow nontrivial shape of theses matrices. 
So both, A TI and hi will be found below microscopically. 

To proceed further we present the Hamiltonian describ- 
ing the TI-SC coupling in a form: 



H — H sc + H2D 
The superconducting part is 



(4) 



H sc - 



d 3 r^f(r) (e sc - M ) * s (r)+ 



J d 3 r ^A^(r)^(r)+A**4.(r)* t (r 



(5) 



where, ^(i) (^fQ.)) are ^ e annihilation (creation) 
operators for the state with the spin up (down), A is 
the superconducting gap, e sc is the single electron kinetic 
energy, and pi is the Fermi energy. 

The second quantization representation for the TI 
Hamiltonian is written in the basis of the Wannier func- 
tions for particles with spin s: 

H ™,s = E C+R,^2A S (R<X,RV) + 

RR',s a-,cr f —a,b 

CT(R,R')<W') c sRV' ( 6 ) 
n R_) is the lattice representation of the 



\) is the lattice representation of the 
% (see Appendix 0). Then H 2D = 



where C2D,. 
BHZ-modei 20112 1 

Y,s H 2D,s- 

Finally, H- lnt reflects the electronic tunneling between 
the SC and TI: 

Hm = J2 E (^rf(R)csR,.+C,RC+ R ^ s (R) 

R.,s a—a,b 
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FIG. 4. (Color online) Bulk and edge states for the inter- 
mediate coupling, a) Energy of the bulk states as function 
of k x ,k y . Without superconductivity, the bulk states di 
persion is isotropic, E(k) = e(k) ± \J ' A 2 k 2 + (M — Bk 2 ) 2 
where k = ^/k 2 + k 2 ,. Superconducting correlations make it 
anisotropic as follows from the noncommutativity of A and/or 
U with H in Eq.(|). b) Gapped edges states. The colors for 
the families of the dispersion curves are the same as in Fig. ^. 
The parameters are chosen as: mkF/{2Trh 2 )t 2 /Al = 1, tj, = 
\/\B- |/|-B+ \t a exp(i7r/6) and the orientation angle ip = n/2. 



where c^^)ji, a is the superposition of |r 6 ,±i), |r 8 ,±i) 
and c^(^)r{, refers to the subband |r6,±|). Integrat- 
ing out the bulk superconductor variables ^ r s (R) using 
the method, developed in Ref. &4 and 25, one obtains 
the effective BdG-Hamiltonian (|j|) for the homogeneous 
tunneling amplitudes i CT R = t a , with the matrix super- 



conducting order parameter and the effective chemical 
potential shift having the form: 



Here 
A T 



A TI = 



mki 



A TI 
. A T 



U = 



<u o 1 
^0 u, 



u 



2irh 2 
m 
2irh 2 a^ 



-1*2 

b a 

tit 



b 

f f* 



tin 
-1*2 



t* a t b 
\tb\ 2 



A TI = p A TI p. 



U = p 1 Up, 



(7) 

(8) 
(9) 



m and /cf are the effective mass and the Fermi momen- 
tum of the bulk superconductor respectively, a TI is the 
characteristic length scale of the order of the lattice con- 
stant in TI. Since A TI <C A, the, proximity induced pa- 
rameters are independent of A.E3 In Ref. p7|-]l9| the po- 
tentials A TI and U were diagonal (trivial) while the off- 
diagonal terms were missed. 

For numerical calculations we take typical parameters: 
A = 3.8 eVA, B = -56.2 eVA 2 , D = -38.7 eVA 2 . 
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FIG. 5. (Color online) Energy landscape for the edge 
states, a) Sketch of the energy landscape along the edge 
(parametrized by the coordinate p). The magnitude of the 
gap may change as a result of spatial fluctuations (like change 
in a shape of the TI boundary or fluctuations in tunneling 
amplitudes) and the state with the energy s would appear 
trapped between the turning points where e < E„ forming 
Andreev bound edge state like it is shown in Fig. |2[ b) Cal- 
culated E g as function of the edge orientation angle <p for the 
ratio t a /tb-\/\B-/B+\ exp(— i7r/3) equal to 1 (upper curve) 
and 2 (bottom curve). Such E g behavior can be observed in 
the sample shaped into a disc, c) Localized edge states in the 
sample shaped into a disc for energy e corresponding to the 
dash-dot line in Fig. Eb. 



Without a loss of generality we take the energy-shift 
parameter C = 0. We do not fix M (— 10meV< 
M < 0) and use it as the energy unit. Our numer- 
ical and analytical calculations show the approximate 
symmetry relation that satisfies the spectrum of H BDG : 

xEQl/x, M/X, ta/Vx, tb/Vx) « #(k, M, t a , t b ), where X 
is a dimensionless scaling parameter. The scaling rela- 
tion appears since M is much smaller than the energy 
scales one can construct form A, B and D. In addition, 
M appears to be the most sensitive to the HgTe layer 
width: it changes with it by several orders of magnitude 
while the other parameters change by ~ 2Q% and their 
changes very slightly modify the spectrumfl 

First we discuss "weak" superconductivity where su- 
perconducting correlations induced in TI can be treated 
perturbatively. In this case matrix elements of A TI and 
U are smaller than the gap in the continuum spectrum, 
M, in the bulk of TI. In the absence of superconducting 
correlations A TI = and U = 0, and there are two elec- 
tron and two hole edge states at each TI surface. The 
edge states have the linear dispersion law with the ve- 
locity s = A\y/B+B-/B\, where B± = B ± D. They 
cross the Fermi energy at k = fco = DM/ (A^J B+BJ) 
and k = — fc , see Fig. ||a. We denote the wave func- 
tions of the electron and hole edge states near k = k n as 
= (^od g c,6,6,6) T andV (2) = (6, 6, Vod g o,6) T , respec- 



tively, where is the zero spinor in the subband space, 
_ e -«W2 ( ^\B7\\ 

^ odso " vw\ {-vm) 

( e -A + r.n _ g-A-r-n) gifcr-1 ^ ^ 

k is the momentum component parallel to the edge, r = 
(x, y), 1, and n are the unit vectors directed along the TI 
boundary and perpendicular to it correspondingly [1 x n 
is aligned with the OZ axis] , and ip is the angle between 1 
and OX axis. The decay length scales of the edge states 
into the bulk of the topological insulators are: 

/ / D \ 2 A 2 M 

where A = A/(2^/B + B-). We stress that spinor com- 
ponents of '(/'edge depend on the Tl-boundary orientation. 




FIG. 6. (Color online) Excitation spectrum in 2D TI for 
the proximity induced potentials being of the same order 
as the gap in TI in the absence of a SC. The gap between 
the branches of the continuum spectrum collapses and TI 
acquires metallic conductivity with the relativistic spectrum 
similar to that in graphene. Solid lines correspond to the 
edge states and the bulk spectrum boundaries in TI with- 
out superconducting correlations. Parameters are chosen as: 
mkF/{2-Kh 2 )tl/M = 12, ip = 0, and t b = ^/\B-\/\B+\t a . 

The dispersion law of the edge states near k = ko 
within the perturbation theory taking in the account the 
superconducting correlations acquires the form: 

ei,a(*) = (#ii+W22±w(fc))/2, (12) 

where lo = y / (2s(fc - fc ) + Uu - U22) 2 + ±Ef, U iu i = 
1 , 2 are the matrix elements of U with respect to the 
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states V> (1 ' 2) and E g = |(A TI )i 
terized through 7± 



They can be parame- 
So, 



T+T-, 



(13) 



become W U ( 22 ) = 



the matrix elements in the see Eq. 
al~£i\, and a = (fci?a TI ) _1 . 

One now sees that the spectrum of the edge states 
becomes dependent upon the orientation of the boundary 
orientation with respect to crystallographic axis. This 
resembles the spectrum that often appear-in the 3D TI 
which that are referred to as "strong" TI.E3 

There is a wealth of the possible coupling-induced be- 
haviours of the edge states energy spectrum. If 7+ = 
or T- = 0, then E g — as well; the situation where 
Eg is very small is also common. A particular picture 
depends on the edge orientation angle <p and/or on the 
tunneling amplitudes, t a and Shown in the Fig. || is 
the situation where at one boundary of the Tl-strip the 
edge states remain gapless (E g — 0) while at the oppo- 
site boundary E g ^ and the edge states have the gap. 
The stripe within which the edge states are confined has 
a finite length (see in Fig. ||), there are points at the edge 
where the Tl-boundary changes its direction and, at the 
same time, the value of E g changes. At these "turning 
points" electron and hole edge (going in the opposite di- 
rection) states with the energy smaller than E g undergo 
the Andreev reflection and form the bound Andreev edge 
state, see Figs. [|[| Illustrated in Figs. ||-[| is the struc- 
ture of the the edge state energy levels in the case where 
E g is finite. 

Now we discuss a general nonperturbative situation. 
The excitation spectrum in 2D TI accounting for the 
proximity induced superconducting correlations in the 
case where proximity induced potentials in TI are of the 
same order as the gap in TI without superconductor on 
top is shown in Fig. ^. The gap between the branches of 
the continuum spectrum nearly closes and TI acquires ef- 
fectively metallic conductivity with the relativistic spec- 
trum similar to that in graphene. Solid lines correspond 
to the edge states and bulk spectrum boundaries in TI 
without superconducting correlations. 



and demonstrated that the s-wave isotropic superconduc- 
tor placed on top of CdTe layer induces superconducting 
correlations in the TI revealing the built in anisotropy 
of TI which did not affect the spectrum when supercon- 
ducting correlations were absent. The form of the edge 
states spectrum essentially depends on the edge orienta- 
tion with respect to crystallographic directions of the TI. 
Depending on the coupling between the superconductor 
and 2D TI, different scenarios can be realized: (i) the 
edge states of the topological insulator acquire a gap, (ii) 
the edge states hybridize into the Andreev localized edge 
state and/or (iii) the gap separating the continuum and 
the edge modes collapses and TI becomes the narrowgap 
(anisotropic) semiconductor. Our predictions can be ver- 
ified by means of, for example, scanning tunnelling spec- 
troscopy measurements of the spectra showed in Fig. ^ 
where the shift of the zero point U\\ + U22 can be tuned 
by the gate placed on top of the CdTe layer. 

Note: After this work has been completed we became, 
aware of the recent experiments-pn InAs/GaSb QW.Ej 
which revealed the 2D TI state.c3 Since this novel TI is 
expected to be well described by the BHZ-model, our 
results apply to InAs/GaSb QW coupled to s-wave su- 
perconductor as well. 
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Appendix A: Lattice model 



III. CONCLUSIONS 

To conclude, we investigated topologically protected 
edge states in QW of HgTe sandwiched between CdTe 



Microscopic description of the coupling between TI and 
the superconductor developed on the base of the standard 
lattice regularization of the BHZ-model (^) replacing its 
parameters by:B 



gfe = C — 2Da 2 [2 — cos k x a — cos k y a] , (Al) 
d = (Aa" 1 sin k x a, — Aa^ 1 sin k y a, M — Ba~ 2 [2 — cos k x a — cos k y a]) . (A2) 

This corresponds to the quadratic Bravais lattice (with the translation vectors a! = ax , a 2 = ay ) with two type 
of states (corresponding annihilation operators are c a R and c&r.) on each site replying to subband states. Therefore 
the Hamiltonian describing TI in the second quantization representation in the basis of the Wannier functions for 
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particles with spin s, takes the form: 



RR'.s <7,a'=a,b 



(A3) 



where 

e2£»,s (Rct,R'ct) = <5r.R' 



M - 



4B 



AD 



AB AD 



(<SR+a„R' + <$R-ai,R' + ^R+a 2 ,R' + <5R-a 2 ,R') (A4a) 



A 



e 2 ZXt( Ra : R ' & ) = -£2D,^(R&, R'fl) = ^ (^R+a 2 ,R - ^R-a 2 ,R' - ^R+a^R' + ^R-ai,R') 

£2ixt( R ^ Ttl'a) = — £223^(Ra, R'6) = — e^^^Ra, R'6) . 



(A4b) 
(A4c) 
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